Random Forest Analysis of Heart Failure Dataset
Methods
- single classification tree.
- Random Forest uses multiple classification trees.
Methods
The Gini Index is referred to as a measure of node purity (James et al. 2021). It can also be used to measure the importance of each predictor. The Gini Index is defined by the following formula where K is the number of classes and \({\hat{p}_{mk}}\) is the proportion of observations in the mth region that are from the kth class. A Gini Index of 0 represents perfect purity.
\[D=-\sum_{n=1}^{K} {\hat{p}_{mk}}(1-\hat{p}_{mk})\]
Bagging is the aggregation of the results from each decision tree. It is defined by the following formula where B is the number of training sets and \(\hat{f}^{*b}\) is the prediction model. Although bagging improves prediction accuracy, it makes interpreting the results harder as they cannot be visualized as easily as a single decision tree (James et al. 2021).
\[{\hat{f}bag(x) = 1/B \sum_{b=1}^{B}\hat{f}^{*b}(x)}\]
Data Used
- The dataset comes from the Faisalabad Institute of Cardiology and the Allied Hospital in Faisalabad Pakistan.
- There are 299 patient records with 13 features per record.
- Random Forest model used for predicting heart failure events for patients.
Variable Importance Plot
FourFold Plot (Confusion Matrix)
Confusion Matrix Heatmap
Variable Correlation Heatmap
ROC - Default Values
trainControl() - Random Selection
trainControl() - Grid Search
Variable Importance Plot - Tuned
FourFold Plot (Confusion Matrix)
Kappa Plot
Tuned vs. Default ROC
Model Results
- The testing accuracies of both models are slightly lower than their respective training accuracies which indicates overfitting of the training dataset.
- The performance of model 2 is better than model 1. This is based off of the model’s training and testing accuracies; precision; F1; and balanced accuracy as they are all higher than those of the default model.